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Executive Summary 

Mesoscale weather conditions can have an adverse effect on space launch, landing, and 
ground processing at the Eastern Range (ER) in Florida and Wallops Flight Facility (WFF) in 
Virginia. During summer, land-water interactions across Kennedy Space Center (KSC) and Cape 
Canaveral Air Force Station (CCAFS) lead to sea and river breeze front formation, which can 
spawn deep convection that can hinder operations and endanger personnel and resources. Many 
other weak locally driven low-level boundaries and their interactions with the sea breeze front and 
each other can also initiate deep convection in the KSC/CCAFS area. All these subtle weak 
boundary interactions often make forecasting of operationally important weather very difficult at 
KSC/CCAFS during the summer. These convective processes often build quickly, last a short 
time (60 minutes or less), and occur over small distances, all of which also pose a significant 
challenge to the local forecasters. Surface winds during the transition seasons (spring and fall) 
pose the most difficulties for the forecasters at WFF. They also encounter problems forecasting 
convective activity and temperature during those seasons. 

Global and national scale models cannot properly resolve important local-scale weather 
features at each location due to their horizontal resolutions being much too coarse. Therefore, a 
properly tuned model at a high resolution is needed to provide improved forecasting capability. 
To accomplish this, the ER and WFF supported the Applied Meteorology Unit (AMU) task to 
perform a number of sensitivity tests in order to determine the best model configuration for 
operational use at each of the ranges to best predict winds, precipitation, and temperature. That 
task was completed in January 2013. This task is a continuation of that work and will provide a 
recommended local data assimilation (DA) and numerical forecast model design optimized for the 
ER and WFF to support space launch activities and for local weather challenges at both ranges. 

The AMU examined different model configurations by running the DA on varying grid 
resolutions and nesting configurations to determine the impact on model skill. Data assimilation 
is an important component in producing quality numerical weather prediction forecasts. The DA 
software chosen for this task was the Gridpoint Statistical Interpolation (GSI) system developed 
by the National Centers for Environmental Prediction. The NASA Short-term Prediction Research 
and Transition Center provided a set of Perl scripts that run both the GSI DA system and the 
Weather Research and Forecast (WRF) model in a compact system. The objective of the scripts 
is to provide an easy-to-use interface for users to be able to execute GSI and WRF. 

The AMU assessed model skill by using an objective statistical analysis that included the 
mean error, root mean square error, and Pearson correlation coefficient. Precipitation forecasts 
were evaluated using the Method for Object-Based Diagnostic Evaluation, a technique developed 
at the National Center for Atmospheric Research. Results indicate that for both the ER and WFF, 
a triple nest configuration that had a 9-km outer, 3-km middle, and 1-km inner nest was the optimal 
model configuration for both ranges. However, a double-nest configuration (2-km outer and 0.67- 
km inner nest) performed the best in predicting precipitation over the ER. Being able to predict 
summertime convection over the ER is operationally important and, for this reason, the AMU 
believes the double-nest configuration can be used as an alternative optimal configuration over 
the ER. 

Future work will provide real-time output of the GSI/WRF cycled runs on the AMU’s Advanced 
Weather Interactive Processing System II workstations. The AMU will also explore running the 
system in a rapid-refresh mode and will explore ensemble modeling to further improve model 
output. 
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1 


Introduction 


Mesoscale weather conditions can have an adverse effect on space launch, landing, and 
ground processing at the Eastern Range (ER) in Florida and Wallops Flight Facility (WFF) in 
Virginia. During summer, land-sea interactions across Kennedy Space Center (KSC) and Cape 
Canaveral Air Force Station (CCAFS) lead to sea breeze front formation, which can spawn deep 
convection that can hinder operations and endanger personnel and resources. Many other weak 
locally driven low-level boundaries and their interactions with the sea breeze front and each other 
can also initiate deep convection in the KSC/CCAFS area. Some of these other boundaries 
include the Indian River breeze front, Banana River breeze front, outflows from previous 
convection, horizontal convective rolls, convergence lines from other inland bodies of water such 
as Lake Okeechobee, the trailing convergence line from convergence of sea breeze fronts due to 
the shape of Cape Canaveral, frictional convergence lines from the islands in the Bahamas, 
convergence lines from soil moisture differences, convergence lines from cloud shading, and 
others. All these subtle weak boundary interactions often make forecasting of operationally 
important weather very difficult at KSC/CCAFS during the convective season (May-Oct). These 
convective processes often build quickly, last a short time (60 minutes or less), and occur over 
small distances, all of which also poses a significant challenge to the local forecasters who are 
responsible for issuing weather advisories, watches, and warnings. Surface winds during the 
transition seasons of spring and fall pose the most difficulties for the forecasters at WFF. They 
also encounter problems forecasting convective activity and temperature during those seasons. 
Therefore, accurate mesoscale model forecasts are needed to aid in their decision making. 

Both the ER and WFF would benefit greatly from high-resolution mesoscale model output to 
better forecast a variety of unique weather phenomena. Global and national scale models cannot 
properly resolve important local-scale weather features at each location due to their horizontal 
resolutions being much too coarse. Therefore, a properly tuned model at a high resolution is 
needed to provide improved capability. This task is a multi-year effort in which the Applied 
Meteorology Unit (AMU) will tune the Weather Research and Forecasting (WRF) model 
individually for each range. The goal of the first year, the results of which are in this report, was 
to tune the WRF model based on the best model resolution and run time while using reasonable 
computing capabilities. To accomplish this, the ER and WFF supported the tasking of the AMU to 
perform a number of sensitivity tests in order to determine the best model configuration for 
operational use at each of the ranges to best predict winds, precipitation, and temperature 
(Watson 2013). This task is a continuation of that work and will provide a recommended local data 
assimilation (DA) and numerical forecast model design optimized for the ER and WFF to support 
space launch activities. The model will be optimized for local weather challenges at both ranges. 

1.1 Phase I Range Modeling Work 

In Phase I of this task, the ER and WFF supported the tasking of the AMU to perform a number 
of sensitivity tests in order to determine the best model configuration for operational use at each 
of the ranges to best predict winds, precipitation, and temperature. The goal of the first phase, 
the results of which are in Watson (2013), was to tune the WRF model based on the best model 
resolution and run time while using reasonable computing capabilities. 

The AMU compared model forecasts for both the ER and WFF using different WRF model 
dynamical cores, grid configurations, and physical schemes to determine the impact on model 
skill. Different model configurations were tested by varying the dynamical core, grid spacing, 
domain size, and forecast length. This enabled the AMU to determine the optimal configuration 
that allowed for the largest domain size and highest resolution to capture unique weather 
phenomena with the shortest wall-clock run time. Once the configurations were chosen, the AMU 
varied the model physics to determine which produced the best forecasts. The WRF model 
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forecasts were validated using simple statistics that compared locally available observational 
surface and upper-air data to the forecast data. The objective statistical analysis included the 
model bias, mean error (ME), and the root mean square error (RMSE). Precipitation forecasts 
were compared to nationally available rainfall data using the Method for Object-Based Diagnostic 
Evaluation (MODE), a technique developed at the National Center for Atmospheric Research 
(NCAR). 

The AMU ran test cases in the warm and cool seasons at the ER and for the spring and fall 
seasons at WFF. For both the ER and WFF, the Advanced Research WRF (ARW) core 
outperformed the Non-hydrostatic Mesoscale Model core. Results for the ER indicate that the Lin 
microphysical scheme and the Yonsei University (YSU) planetary boundary layer (PBL) scheme 
is the optimal model configuration for the ER. It consistently produced the best surface and upper 
air forecasts, while performing fairly well for the precipitation forecasts. Both the Ferrier and Lin 
microphysical schemes in combination with the YSU PBL scheme performed well for WFF in the 
spring and fall seasons. 

1 .2 Report Format and Outline 

This report presents the findings from a study of local DA and numerical forecast model design 
optimized for the ER and WFF to support space launch and for predicting unique weather 
phenomena at the ER and WFF. This analysis examined different model configurations by running 
the DA on varying grid resolutions and nesting configurations to determine the impact on model 
skill. The AMU assessed model skill by using an objective statistical analysis that included the 
ME, RMSE, and Pearson correlation coefficient (PCC). Precipitation forecasts were evaluated 
using MODE. Section 2 describes the DA software, the observational data used in the study, the 
model configurations, and the NASA Short-term Prediction Research and Transition Center 
(SPoRT) supplied Perl scripts used to run the forecast system. Section 3 describes the forecast 
validation technique and the results for both the ER and WFF. Section 4 contains the conclusions 
and future work. 
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2 Data and Model Configuration 

The important aspects of this study were choice of DA software, the observational data ingest, 
period of record (POR), the DA/model configurations, and the DA/modeling system Perl scripts 
supplied by SPoRT. 

2.1 Data Assimilation Software 

DA is an important component in producing quality numerical weather prediction forecasts. 
DA methods are used to create a best estimate of the state of the atmosphere at the forecast 
initial time using information from observations and a background model forecast. This is 
accomplished through the minimization of an objective function that measures the weighted 
distance of the analysis from the observations and the background model. The weights assigned 
to each term are based on the error characteristics of the observations and the background model 
(Kleist et al. 2009). This creates an initial state that more closely resembles the state of the 
atmosphere at the model initialization time. 

The DA software chosen for this task was the Gridpoint Statistical Interpolation (GSI) system 
developed by the National Centers for Environmental Prediction (NCEP). The GSI system is a 
three-dimensional variational DA system that is a freely available, community system designed to 
be flexible, state-of-the-art, and can run efficiently on various parallel computing platforms. It can 
be applied to both regional and global applications. 

2.2 Observational Data Ingest 

GSI can ingest large quantities of atmospheric observations and has developed capabilities 
for data thinning, quality controlling, and satellite radiance bias correction (Wang 2010). Table 1 
lists some of the conventional and satellite radiance/brightness temperature observations that 
GSI can assimilate. 

Most of these observations are complex and many of them need to be reformatted into Binary 
Universal Form for the Representation of meteorological data (BUFR) format or quality controlled 
before being used by GSI. NCEP produces quality-controlled data in BUFR format, called 
PrepBUFR files that can be used directly in GSI. These freely available files were used for this 
task. 

2.3 Model Configuration and Test Cases 

The AMU determined the main model configuration from the first phase of the Range Modeling 
task. The ARW core was used with the Lin microphysical scheme and the YSU PBL scheme for 
both the ER and WFF. Although the AMU previous work found the optimal horizontal grid spacing 
for the ER was a 2-km outer and 0.67-km inner domain and a 4-km outer and 1.33-km inner 
domain for WFF, the AMU conducted additional testing on the optimal horizontal grid spacing 
once the DA was added. Specifically, different configurations were run to determine the impact 
on the forecasts of running the DA over differing grid resolutions. For the ER, the AMU ran three 
configurations: 

• Single domain: 1-km domain in which the DA was run (referred to as ‘1 dom’, Figure 1), 

• Nested domain: 2-km outer and 0.67-km inner domain in which the DA was run on the 
outer 2-km domain (referred to as ‘2 doms’, Figure 2), 

• Triple nested domain: 9-km outer, 3-km middle, and 1-km inner domain on which the DA 
was run on the 9-km outer domain (referred to as ‘5 doms’, Figure 3). 
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For WFF, the AMU ran two configurations: 

• Nested domain: 4-km outer and 1.33-km inner domain in which the DA was run on the 
outer 4-km domain (referred to as ‘2 doms’, Figure 4), 

• Triple nested domain: 9-km outer, 3-km middle, and 1-km inner domain on which the DA 
was run on the 9-km outer domain (referred to as ‘5 doms’, Figure 3). 


All other parameters were the same for each model run. 


Table 1. List of conventional and satellite observations that can be assimilated into GSI. 

Conventional observations 

• Radiosondes 

• Pibal winds 

• Synthetic tropical cyclone 

• Conventional aircraft 

• ASDAR aircraft reports 

winds 

reports 

• GMS, JMA, and 

• MDCARS aircraft reports 

• MODIS IR and water vapor 

METEOSAT cloud drift IR 

• EUMETSAT and GOES 

winds 

and visible winds 

water vapor cloud top winds 

• Surface land observations 

• SSM/I wind speeds 

• QuikScat, ASCAT and 

• Doppler radial velocities 

• VAD (NEXRAD) winds 

OSCAT wind speed and 

• SBUV ozone profiles, MLS 

• SST 

direction 

(including NRT) ozone, and 

• Doppler wind Lidar data 

• GPS precipitable water 

OMI total ozone 

• Tail Dopppler Radar radial 

estimates 

• Wind profilers: US, JMA 

velocity and super- 

• Tropical storm VITAL 

• Dropsondes 

observation 

• Radar radial wind and 

• GEOS hourly IR and coud 

• METAR cloud observations 

reflectivity Mosaic 

top wind 

• SSM/I and TRMM TMI 

• PM2.5 

• GPS Radio occultation 

precipitation estimates 

• Surface ship and buoy 

refractivity and bending 


observation 

angle profiles 



Satellite radiance/brightness temperature observations 

• SBUV: n17, n18, n19 

• HIRS: metop-a, metop-b, 

• GOESJMG: gl 1 , g12 

• AIRS:aqua 

n17, n19 

• AMSU-A: metop-a, metop- 

• MHS: metop-a, metop-b, 

• SSMI: f14, f 1 5 

b, n15, n18, n19, aqua 

n18, n19 

• SNDR: gl 1 , g12, g13 

• AMSU-B: metop-b, n17 

• AMSRE: aqua 

• OMI: aura 

• SSMIS: f16 

• GOME: metop-a, metop-b, 

• SEVIRI: m08, m09, mIO 

• IASI: metop-a, metop-b 

• ATMS: NPP 
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Figure 1. Map of the ER showing the single 1-km (D01) model domain 
boundary (1 dom). 



Figure 2. Map of the ER showing the nested 2-km outer (D01) and 
0.67-km inner (D02) model domain boundaries (2 doms). 


9 





Figure 3. Map of the triple nest configuration showing the 9-km outer 
(D01), 3-km middle (D02 and D04), and 1-km inner (D03 and D05) 
model domain boundaries over the ER and WFF (5 doms). 
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Figure 4. Map of WFF showing the nested 4-km 
outer (D01 ) and 1 ,33-km inner (D02) model domain 
boundaries (2 doms). 




The AMU ran each DA/model simulation at the specified horizontal resolution with 35 
irregularly spaced, vertical sigma levels up to 50 mb. Each run was initiated four times per day at 
0000, 0600, 1200, and 1800 UTC and integrated 12 hours using the 13-km Rapid Refresh (RAP) 
model for boundary conditions and as the background model first-guess field, Land Information 
System (LIS) data from SPoRT for land surface data, and sea surface temperature (SST) data 
from both NCEP’s Real-time Global SSTs and the SPoRT 2-km SST composites. Initial conditions 
were created using a cycled GSI/WRF approach that is described in Section 2.4. The POR for 
the test cases was from 1200 UTC 27 August 2013 to 0600 UTC 10 November 2013. 

2.4 NASA SPoRT Perl Scripts 

SPoRT provided a set of Perl scripts that run both the GSI DA system and the WRF model in 
a compact system. The objective of the scripts is to provide an easy-to-use interface for users to 
be able to execute GSI and WRF. The SPoRT scripts are simply a wrapper to support running of 
the software systems, which must be downloaded and installed separately by the user. 

The GSI/WRF scripts use a cycled GSI system similar to the operational North American 
Mesoscale (NAM) model. The scripts run a 12-hour pre-cycle in which data are assimilated from 
1 2 hours prior up to the model initialization time. This is done due to the time latency of the satellite 
data. Satellite data are not available instantaneously as it takes time to receive and process. If 
the pre-cycling did not occur, there would be very little influence on model output from the satellite 
observations, which have been shown to have the largest positive impact on most forecast 
systems. Once the pre-cycling is complete, a 12-hour forecast is run. 

A schematic of the GSI/WRF pre-cycling system is shown in Figure 5. Model initialization 
times are shown in green, pre-cycle times are shown in blue, and forecasts are shown in red. As 
stated above, the model is run four times per day. For a model initialization time of 0000 UTC 
(tOOZ in Figure 5), the GSI/WRF scripts are started between 0000 and 0100 UTC. The scripts 
begin their pre-cycling 12 hours prior to initialization time, in this case at 1200 UTC of the previous 
day. Initial conditions for the WRF forecast are first created by processing the RAP data as the 
background model, the LIS land surface data, and NCEP and SPoRT SST data using the WRF 
preprocessing system. Once a background grid has been established, observational data are 
assimilated into the background grid using the GSI system. After the observations have been 
assimilated, a 3-hour WRF forecast is run (red arrow between tm12 and tm09 for tOOZ). The pre- 
cycle begins again using data from nine hours prior to the initialization time (tm09, 1500 UTC of 
the previous day), however, the 3-hour WRF forecast is now used as the background model for 
the next forecast. This process continues every three hours until the initialization time is reached. 
At 0000 UTC after the background grid is created using the 3-hour WRF forecast and the 
observational data is assimilated, a 12-hour WRF forecast is run (red dashed line). The process 
repeats itself every six hours. 
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Figure 5. Schematic showing the 12-hour pre-cycling that occurs in the GSI/WRF 
system. Text in green is the model initialization time, solid red arrows are the pre- 
cycle forecast, and red dashed arrows are the full model forecast. This figure is a 
recreation of a NASA SPoRT produced schematic. 
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3 Model Forecast Validation 
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The AMU validated the GSI/WRF model forecasts using statistics that compared locally 
available surface observations to the forecast data. Precipitation forecasts were compared to 
nationally available rainfall data using a technique developed at NCAR. 


3.1 Observational Data 

In order to verify the GSI/WRF model performance, surface weather observations of 
temperature, dewpoint, wind speed and direction, and atmospheric pressure were required. 
NCEP’s Meteorological Assimilation Data Ingest System (MADIS) and Stage IV precipitation data 
were used for the observational datasets. 

MADIS was developed by NOAA’s Earth System Research Laboratory Global Systems 
Division to collect, integrate, quality control, and distribute observations from a multitude of 
organizations. MADIS includes various meteorological surface datasets (such as Meteorological 
Aerodrome Report (METAR), maritime, and mesonet data, among others), profiler network data, 
aircraft data, radiosondes, satellite data, hydrological surface data, radiometer, and snow data. 
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Figure 6. Map of a sampling of the mesonet and METAR weather 
station locations over the ER and the surrounding areas from MADIS. 
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For this task, the AMU downloaded METAR and mesonet data to validate the GSI/WRF 
forecasts. METAR is the international standard code format for hourly surface weather 
observations and typically contains data for the temperature, dew point, wind speed and direction, 
precipitation, cloud cover and heights, visibility, and barometric pressure. Mesonet data is a 
network of automated weather stations designed to observe mesoscale meteorological 
phenomena, such as dry lines, squall lines, and sea breezes, and report conditions in time 
intervals anywhere from 1 to 15 minutes. The locations of some of the METAR and mesonet 
stations over the ER and WFF are shown in Figure 6 and Figure 7, respectively. 
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Figure 7. Map of a sampling of the mesonet and METAR weather 
station locations over WFF and the surrounding areas from MADIS. 
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To verify precipitation, the AMU compared hourly forecast rainfall accumulation to the NCEP 
Stage-IV analysis ( http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/staqe4/ ). This analysis 
combines radar data and rain gauge reports to produce hourly rainfall accumulation on a 4-km 
grid. It is a manually quality-controlled continental U.S. mosaic from the regional 1-hour 
precipitation analyses produced by 12 National Weather Service River Forecast Centers (Lin and 
Mitchell 2005). 

3.2 MET Software 

For the objective analysis, the AMU compared observed wind speed, temperature, dewpoint 
temperature, mean sea-level pressure, and accumulated precipitation observations to the 
forecast variables using the latest version of the Model Evaluation Tools (MET) software. This 
software was developed by the NCAR Developmental Testbed Center. It is a state-of-the-art suite 
of verification tools that uses output from the WRF model to compute standard verification scores 
comparing gridded model data to point or gridded observations, and uses spatial verification 
methods comparing gridded model data to gridded observations using object-based 
decomposition procedures. For this task, two MET tools were chosen to validate the GSI/WRF 
forecasts: the Point-Stat tool and the MODE tool. 

3.2.1 Point-Stat Tool 

The MET Point-Stat tool was used to verify the surface forecasts. This tool provides 
verification statistics for forecasts at observation points, in this case, at the METAR and mesonet 
point observations. The Point-Stat tool matches gridded forecasts to point observation locations 
using a user-specified interpolation approach and computes the verification statistics. 

Many output statistics are available within the Point-Stat tool. This study looked at three of 
those statistics for wind speed, temperature (T), dewpoint temperature (Td), and mean sea-level 
pressure (MSLP): the ME, the RMSE, and the PCC; and two statistics for wind direction: the ME 
and the RMSE. The statistics compared all mesonet and METAR observations available to the 
corresponding locations in the model forecast output at a 1-hour interval. 

The mean error, ME, is a measure of the overall bias of the model parameter being compared. 
A perfect forecast has ME = 0. It is defined as: 


n = number of forecast and observation pairs over the forecast period, 
fi = WRF forecast of T, Td, MSLP, wind speed, or wind direction, and 
Oi = observed T, Td, MSLP, wind speed, or wind direction. 

The model RMSE was calculated to measure the magnitude of the error. It is useful in 
determining whether the forecasts produced large errors, as it gives relatively high weight to large 
errors. It is calculated using the following equations: 



where: 


n 



i-1 


RMSE = VMSE 


where n, fi, and o, are defined as above. 
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The PCC, r, measures the strength of the linear association between the forecast and 
observed parameters. It is defined as: 


where n, fj, and o, are defined as above and: 

/ = average forecast over time and space of T, Td, MSLP, wind speed, or wind direction, and 

o = average observed overtime and space of T, Td, MSLP, wind speed, or wind direction. 

The PCC can range between -1 and 1 where 1 indicates a perfect correlation, -1 indicates a 
perfect negative correlation, and 0 indicates no correlation between the forecast and 
observations. For example, the PCC for wind speed measures whether large values of forecast 
wind speed tend to be associated with large values of observed wind speed (positive correlation), 
whether small values of forecast wind speed tend to be associated with large values of observed 
wind speed or vice versa (negative correlation), or whether values of both variables are unrelated 
(correlation near 0). 

3.2.2 MODE 

To verify precipitation, the AMU compared the forecast hourly rainfall accumulation to the 
observed rainfall over the same time period. MODE was used to determine the skill of each model 
configuration. MODE is an object-based verification system that compares gridded observations 
to gridded forecasts. It resolves and compares objects, such as areas of accumulated rainfall, in 
both the forecast and observed fields. The objects are described geometrically and then the 
attributes of the objects can be compared (Davis et al. 2006). MODE outputs statistics that 
describe the correlation between the objects and allows the user to identify forecast strengths or 
weaknesses. Details about how objects are identified and characterized can be found in Davis et 
al. (2006). For this report, the objects of interest are areas of accumulated rainfall. Therefore, 
references to objects are references to areas of resolved accumulated rainfall throughout the 
forecast period. 

Once the objects have been identified, their various properties are evaluated and compared. 
The object attributes examined by the AMU in this task included the centroid distance, area ratio, 
and total interest value. The centroid distance is the vector difference between the centroids of 
the forecast and observed objects. It describes the location bias in the forecasts. The smaller the 
distance between the centroids, the better the forecast. The area ratio compares the area, or 
number of grid squares, the forecast object occupies compared to the observed object. An area 
ratio of 1 is considered a perfect correlation. Interest value is defined as the differences in 
particular attributes between the forecast and observed objects. Interest values of 0 indicate no 
interest, or a poor forecast, while a value of 1 indicates high interest, or a good forecast. The total 
interest value is a weighted sum of specific interest values and is used as an overall indicator of 
the quality of the precipitation forecast. Total interest value is large when forecast and observed 
objects are well correlated (are roughly the same size and are close to each other) and is small 
when they are not well correlated. 


i k-fb-o) 


r = 


i=\ 
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3.3 ER Results 


The AMU validated model performance for the ER using forecasts from the grids described in 
section 2.3, where 2 doms references the nested 2-km outer and 0.67-km inner domain (Figure 
2), 1 dom references the single 1-km domain (Figure 1), and 5 doms references the triple nested 
9-km outer, 3-km middle, and 1-km inner domain over the ER (Figure 3). 

3.3.1 Surface Forecasts 

The AMU validated the GSI/WRF forecasts with the local METAR and mesonet data. Figure 
8 shows the ME for wind speed, wind direction, temperature, dewpoint temperature, and MSLP 
from the three GSI/WRF configurations averaged over each hour of each 12-hour forecast for the 
entire POR at the ER. 

Overall, the ME results indicate that the triple-nest configuration performed the best of the 
three configurations, followed by the nested domain, and then the single domain. The triple-nest 
configuration had the lowest ME for wind speed, wind direction, dewpoint temperature, and MSLP 
while the nested domain had the lowest ME for temperature. The single domain had the highest 
ME for wind speed, temperature, and dewpoint temperature. 

The average hourly ME results for wind speed indicated that all three domain configurations 
over-predicted the wind speed throughout the forecasts. The ME for wind speed for the single 
domain was approximately 2 m/s higher than for either nested domain during the entire forecast 
period with an average hourly ME of approximately 5 m/s. All three domains followed the same 
trend of slightly increasing ME throughout the forecasts for wind direction. The single domain had 
a consistent warm bias throughout the forecasts as indicated by the average hourly ME for 
temperature with a maximum ME of approximately 0.7 K. Both nested domains exhibited a cool 
bias throughout the forecasts, except for the 0-hour forecast. All domain configurations exhibited 
a cool dewpoint temperature bias, with the single domain showing the largest cool bias in the 2 
to 2.3 K range. Results from the average hourly ME for MSLP show that all three configurations 
followed the same trend of an initial negative, or low pressure, bias followed by a slight positive, 
or high pressure, bias. 
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Figure 8. Chart of the average hourly ME for the 12-hour forecast over the entire POR for a) 
wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 
three GSI/WRF configurations at the ER. 

As with ME, the RMSE results (Figure 9) indicate that the triple-nest configuration performed 
the best of the three configurations, followed by the nested domain, and then the single domain. 
The triple-nest configuration had the lowest RMSE for wind speed, wind direction, and dewpoint 
temperature while the nested domain had the lowest RMSE for temperature and MSLP. The 
single domain had the highest RMSE for all variables except wind direction. 
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The general trends for the average hourly RMSE for wind speed and direction are nearly the 
same as the ME. The RMSE for temperature and dewpoint temperature indicate that the single 
domain still performed the worst. Average hourly RMSE for MSLP results are similar to the ME 
results, with all three domain configurations showing nearly the same trends in RMSE throughout 
the forecast. 




Average Hourly Root Mean Square Error - Mean Sea 
Level Pressure 



Figure 9. Chart of the average hourly RMSE for the 12-hour forecast over the entire POR for a) 
wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 
three GSI/WRF configurations at the ER. 
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The PCC can indicate whether the model configurations captured the overall trend of the 
observed variables. Using wind speed as an example, it answers the question of whether the 
model winds fluctuated positively and negatively with the same magnitude as the observed winds. 
The closer the PCC is to 1, the better the model was able to capture these trends. When 
comparing these particular forecast vs. observed variables, only positive coefficients indicate any 
value in the model forecasts. 


The results from the PCC (Figure 10) calculation indicate that the triple-nested configuration 
again performed the best of the three configurations followed by the single domain and then the 
nested domain. The triple-nest configuration had the highest PCC for wind speed, temperature, 
and MSLP while the single domain had the highest PCC for dewpoint temperature. 


All results show a positive correlation, except for the nested domain during forecast hours 8- 
9 for MSLP indicating that the fluctuations in each of the variables was captured by the model. 
However, overall the values for PCC were low. This may be due to the difficulty in capturing 
mesoscale phenomena in the summertime over the ER. 




Figure 10. Chart of the average hourly PCC for the 12-hour forecast over the entire POR for a) 
wind speed, b) temperature, c) dewpoint temperature, and d) MSLP from the three GSI/WRF 
configurations at the ER. 
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3.3.2 Precipitation Forecasts 

The AMU compared precipitation forecasts from all three model configurations to determine 
performance differences. One-hour forecast accumulated rainfall for each hour of each of the 12- 
hour forecasts was compared to the one-hour accumulation of observed rain using the NCEP 
Stage-IV analysis data for each day during the POR. The POR summary statistics for each hour 
of the forecast of centroid distance, area ratio, and total interest value from the MODE software 
are shown in Figure 11. The centroid distance (km) results indicate that the nested domain 
precipitation matched the location of the observed precipitation most closely throughout the 
forecasts, followed by the triple-nest configuration and then the single domain. The area ratio for 
the nested domain indicates that the forecast most closely matched the areal coverage of 
observed precipitation, with the triple-nest and single domain performing slightly worse and 
similarly, respectively. Interest value functions near 0.9 for the nested domain indicates that 
overall the forecast correlated the best with the observed precipitation, followed by the triple-nest 
configuration and then single domain. Overall, the nested domain outperformed both triple-nest 
and single domain configurations. 




Figure 1 1 . Chart of the average hourly a) centroid distance (km), b) area ratio, and c) total interest 
value from the three GSI/WRF configurations for each hour of the 1 2-hour forecast over the entire 
POR at the ER. 
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3.4 WFF Results 


The AMU validated model performance for WFF using forecasts from the grids described in 
section 2.3, where 2 doms references the nested 4-km outer and 1 ,33km inner domain (Figure 4) 
and 5 doms references the triple-nested 9-km outer, 3-km middle, and 1-km inner domain over 
WFF (Figure 3). 

3.4.1 Surface Forecasts 

The AMU validated the GSI/WRF forecasts with the local METAR and mesonet data. Figure 
12 shows the ME for wind speed, wind direction, temperature, dewpoint temperature, and MSLP 
from the two GSI/WRF configurations averaged over each hour of each 12-hour forecast for the 
entire POR at WFF. 

Overall, the ME results indicate that the triple-nest configuration performed better than the 
nested domain. The triple-nest configuration had the highest ME for wind speed, wind direction, 
temperature, and MSLP while the nested domain had the highest ME for dewpoint temperature. 

Both model configurations over-predicted the wind speed throughout the forecast with a large 
increase in ME after the first forecast hour. However, the error for the triple-nest configuration was 
nearly 1 .5 m/s less than that for the nested domain throughout the forecast. The same large 
increase after the first forecast hour in ME for wind direction was present in the nested domain 
with slightly decreasing values of ME after forecast hour 3, while the triple-nest configuration 
showed gradual increasing ME error values throughout the forecast period. The triple-nest 
configuration had lower ME values than the nested domain with differences in ME of 
approximately 15 to 30 degrees. ME values for temperature were nearly the same for both 
configurations during the first five forecast hours, after which the triple-nest configuration 
outperformed the nested domain with ME values of nearly 0. The triple-nest configuration 
consistently forecasted dewpoint temperature values that were too cool, while the nested domain 
was too cool during the first 5 hours and then too warm after that. Both configurations followed 
the same ME trend for MSLP with pressures that were initially too low and then too high in the 
latter part of the forecasts. 
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Average Hourly Mean Error - Mean Sea Level Pressure 



Figure 12. Chart of the average hourly ME for the 12-hour forecast over the entire POR for a) 
wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 
two GSI/WRF configurations at WFF. 

Similar to the ME, the RMSE results (Figure 13) indicate that the triple-nest configuration 
performed the best for all variables. The RMSE wind speed and direction charts are nearly 
identical to the ME error charts indicating that there were not any large outliers for forecasted wind 
speed or direction. The triple-nest configuration RMSE for temperature and dewpoint temperature 
was roughly 1 .25 K lower than that for the nested domain throughout the forecasts. Except for 
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forecast hours 2 and 3, the triple-nest configuration outperformed the nested domain for MSLP 
with differences of approximately 60 Pa lower in the latter half of the forecasts. 





Figure 13. Chart of the average hourly RMSE for the 12-hour forecast over the entire POR for a) 
wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 
two GSI/WRF configurations at WFF. 
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The results from the PCC calculation (Figure 14) indicate that the triple-nest configuration 
again outperformed the nested domain for wind speed, temperature, dewpoint temperature, and 
MSLP. The PCC for the triple-nest configuration for wind speed and temperature was 
approximately 0.05 to 0.1 higher than the nested domain throughout the forecast. The PCC for 
dewpoint temperature was 0.2 to 0.3 higher for the triple-nest configuration and was up to 0.1 
higher through forecast hours 4 to 1 1 than the nested domain for MSLP. It is interesting to note 
that the highest PCC values for WFF were nearly double that of the ER. During the summer and 
fall, different systems drive the local weather at each range. There are more synoptic systems 
that influence the weather over the WFF during summer and fall while in the summer, the weather 
over the ER is locally driven by mesoscale phenomena. In general, models will perform better 
when the local weather is driven by larger-scale synoptic systems. Therefore, it is expected that 
the GSI/WRF system performed better for WFF than for the ER during the POR. 




Figure 14. Chart of the average hourly PCC for the 12-hour forecast over the entire POR for a) 
wind speed, b) temperature, c) dewpoint temperature, and d) MSLP from the two GSI/WRF 
configurations at WFF. 
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3.4.2 Precipitation Forecasts 

The AMU compared precipitation forecasts from both model configurations to determine 
performance differences. One-hour forecast accumulated rainfall for each hour of each of the 1 2- 
hour forecasts was compared to the one-hour accumulation of observed rain using the NCEP 
Stage-IV analysis data for each day during the POR. The POR summary statistics for each hour 
of the forecast of centroid distance, area ratio, and interest function from the MODE software are 
shown in Figure 15. The centroid distance (km) results indicate that forecasts from both 
configurations were very similar in their ability to match the location of the observed precipitation 
throughout the forecasts. However, the triple-nest configuration did slightly better in predicting the 
precipitation location. The area ratio results indicate that accumulated precipitation for the triple- 
nest configuration matched the areal coverage of observed precipitation more closely, although 
the results were comparable. Total interest values were very similar for both configurations with 
values near 0.8. Overall, the triple nest configuration very slightly outperformed the nested 
domain. 




Figure 15. Chart of the average hourly a) centroid distance (km), b) area ratio, and c) total interest 
value from the two GSI/WRF configurations for each hour of the 12-hour forecast over the entire 
PORatWFF. 
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4 Conclusions and Future Work 

This report summarizes the findings from the AMU task to determine a recommended local 
DA and numerical forecast model design optimized for the ER and WFF to support space launch 
activities and for local weather forecasting challenges at each range. The AMU ran the GSI/WRF 
model system over part of the summer and fall seasons for each range while varying grid 
resolutions on which the DA was run and varying the nesting configurations to determine the 
impact on model skill. In general for both the ER and WFF, the triple-nest configuration 
outperformed the other configurations. However, although the results for the ER indicate that the 
triple-nest configuration performed best for most variables as evidenced by the ME, RMSE, and 
PCC, the nested configuration did the best in predicting precipitation for the ER. Summertime 
convection over the ER is an important meteorological variable to predict and, for this reason, it 
is the AMU’s recommendation to use either the nested or triple-nest configuration as the optimal 
model configuration for the ER. The triple-nest configuration performed the best for nearly all 
variables at WFF. Wind, temperature, and convective activity forecasts during the fall and spring 
seasons pose the most difficulties for forecasters at WFF. Therefore, it is the AMU’s opinion that 
the triple-nested domain is the optimal model configuration for WFF. 

The AMU has been tasked with a follow-on modeling effort to: 

• Port GSI/WRF code from the AMU developmental cluster to the AMU real-time cluster 
and to run every three hours, 

• Display real-time output of the GSI/WRF cycled runs on the AMU’s AWIPS II 
workstations, 

• Work with SPoRT to determine if the GSI/WRF system can be run in a rapid-refresh 
mode. If so, determine the time-frame to set up the rapid-refresh system and 
implement if possible within the time-frame of this task, and 

• Explore ensemble modeling using the WRF model and determine the level of effort 
and time-frame to set up an ensemble modeling system. 

This follow-on task will benefit operations by providing high spatial and time resolution forecasts 
from a model tuned specifically for their location. It will also continue to improve the local model 
by incorporating techniques that have been shown to enhance model performance. 
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List of Acronyms 


45 WS 

45th Weather Squadron 

MSLP 

Mean Sea-Level Pressure 

AMU 

Applied Meteorology Unit 

NAM 

North American Mesoscale 

ARW 

Advanced Research WRF 

NCAR 

National Center for 

AWIPS 

Advanced Weather Interactive 


Atmospheric Research 


Processing System 

NCEP 

National Centers for 

CCAFS 

Cape Canaveral Air Force 


Environmental Prediction 


Station 

NMM 

Non-hydrostatic Mesoscale 

DA 

Data Assimilation 


Model 

ER 

Eastern Range 

PBL 

Planetary Boundary Layer 

GSI 

Gridpoint Statistical 

PCC 

Pearson Correlation Coefficient 


Interpolation 

POR 

Period of Record 

KSC 

Kennedy Space Center 

RMSE 

Root Mean Square Error 

LIS 

Land Information System 

RAP 

Rapid Refresh 

MAD IS 

Meteorological Assimilation 
Data Ingest System 

SPoRT 

Short-term Prediction 
Research and Transition 

ME 

Mean Error 


Center 

MET 

Model Evaluation Tools 

SST 

Sea Surface Temperature 

METAR 

Meteorological Aerodrome 

WFF 

Wallops Flight Facility 


Report 

WRF 

Weather Research and 

MODE 

Method for Object-Based 


Forecasting 


Diagnostic Evaluation 

YSU 

Yonsei University 
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NOTICE 


Mention of a copyrighted, trademarked or proprietary product, service, or document does not 
constitute endorsement thereof by the author, ENSCO Inc., the AMU, the National Aeronautics 
and Space Administration, or the United States Government. Any such mention is solely for the 
purpose of fully informing the reader of the resources used to conduct the work reported herein. 
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